Wearable Capacitive-based Wrist-worn Gesture Sensing System

Gesture control plays an increasingly significant role in modern human-machine interactions. This paper presents an innovative method of gesture recognition using flexible capacitive pressure sensor attached on user’s wrist towards computer vision and connecting senses on fingers. The method is based on the pressure variations around the wrist when the gesture changes. Flexible and ultrathin capacitive pressure sensors are deployed to capture the pressure variations. The embedding of sensors on a flexible substrate and obtain the relevant capacitance require a reliable approach based on a microcontroller to measure a small change of capacitive sensor. This paper is addressing these challenges, collect and process the measured capacitance values through a developed programming on LabVIEW to reconstruct the gesture on computer. Compared to the conventional approaches, the wrist-worn sensing method offerings a low-cost, lightweight and wearable prototype on the user’s body. The experimental result shows that the potentiality and benefits of this approach and confirms that accuracy and number of recognizable gestures can be improved by increasing number of sensor

Physical reservoir computing with dynamical electronics

Since the advent of data-driven society, mass information generated from human activity and the natural environment has been collected, stored, processed, and then dispersed under conventional von Neumann architecture. However, further scaling the computing capability in terms of speed and power efficiency has been significantly slowed down in recent years due to the fundamental limits of transistors. To meet the increasingly demanding requirement for data-intensive computation, neuromorphic computing is a promising field taking the inspiration from the human brain, an extremely efficient biological computer, to develop unconventional computing paradigms for artificial intelligence. Reservoir computing, a recurrent neural network algorithm invented two decades ago, has received wide attention in the field of neuromorphic computing because of its unique recurrent dynamics and hardware-friendly implementation schemes. Under the concept of reservoir computing, hardware’s intrinsic physical behaviours can be explored as computing resources to keep the machine learning within the physical domain to improve processing efficiency, which is also known as physical reservoir computing. This thesis focuses on modelling and implementing physical reservoir computing based on dynamical electronics, along with its applications with sensory signals. First, the fundamental of the reservoir computing algorithm is introduced. Second, based on the reservoir algorithm and its functionalities, two different architectures for physically implementing reservoir computing, delay-based reservoir and parallel devices, are investigated to perform temporal signal processing. Thirdly, an efficient implementation architecture, namely rotating neurons reservoir, is developed. This novel architecture is evaluated in both theoretical analysis and experiments. An electrical prototype of the rotating neurons reservoir exhibits unique advantages such as resource-efficient implementation and low power consumption. More importantly, the theory of rotating neurons reservoir is highly universal, indicating that a rotational object embedded with dynamical elements can act as a reservoir computer

Wrist-worn gesture sensing with wearable intelligence

This paper presents an innovative wrist-worn device with machine learning capabilities and a wearable pressure sensor array. The device is used for monitoring different hand gestures by tracking tendon movements around the wrist. Thus, an array of PDMS-encapsulated capacitive pressure sensors is attached to the user to capture wrist movement. The sensors are embedded on a flexible substrate and their readout requires a reliable approach for measuring small changes in capacitance. This challenge was addressed by measuring the capacitance via the switched capacitor method. The values were processed using a programme on LabVIEW to visually reconstruct the gestures on a computer. Additionally, to overcome limitations of tendon’s uncertainty when the wristband is re-worn, or the user is changed, a calibration step based on the Support Vector Machine (SVM) learning technique is implemented. Sequential Minimal Optimization (SMO) algorithm is also applied in the system to generate SVM classifiers efficiently in real-time. The working principle and the performance of the SVM algorithms demonstrate through experiments. Three discriminated gestures have been clearly separated by SVM hyperplane and correctly classified with high accuracy (>90%) during real-time gesture recognition

A Delay-Based Neuromorphic Processor for Arrhythmias Detection

Cardiovascular disease is the leading cause of global mortality, with 17.5 Million deaths per annum (World Health Authority, WHO). Innovative hardware based cardiac recording devices could help elevate this burden. Delay-based reservoir computing is a novel computational framework with only a single nonlinear node. This feature makes it a strong candidate for the hardware implementation of an analogue cognitive system. Such a system can be exploited to improve the energy efficiency of data processing in implantable bioelectronic devices. This paper presents a system modelling of this network that is capable of cognitively processing Electrocardiograph (ECG) signals from the MIT-BIH arrhythmia database. The proposed single-input single-output model receives an encoded ECG signal while the output amplitude pattern aids the diagnostic interpretation. The information processor is an analogue circuit with the dynamic properties of Mackey-Glass nonlinearity and fading memory. To validate this system and mimic real-time operation, the simulation is designed to detect ventricular ectopic beats, an ectopic heartbeat type, using a continuous ECG signal without any signal segmentation or feature extraction. After training, the model successfully locates ventricular ectopic beat with 87.51% sensitivity and 94.12% accuracy for the testing dataset from three patients

Eye Tracking Simulation for a Magnetic-Based Contact Lens System

In this paper, we present a simulation of an eye motion tracking system. The system consists of a moving magnet and three static magnetic sensors, which implies the magnet embedded in a contact lens and sensors fixed on the spectacles in the application scenario. When the eye is moving, the changing relative position between sensors and magnet will result in different sensory outputs that encodes eye movement information. The simulation of eye movements and corresponding magnetic fields was carried out in MATLAB MathWorks software. After collecting the sensory output, artificial neural network was used to decode the signal and classify the direction of gaze. In total 30 different configurations were tested to determine which one gives the highest accuracy. The network prior to each configuration was trained and the output was compared to the actual position of the eye. It was found that the lens misplacement may cause a lot of issues and requires further investigation to lower its impact on the results. This could be fixed by introducing a calibration step. For all of the configurations, usually, the confusion occurred on the neighbouring classes. This could be due to poor design of the classes, where borders of the regions do not overlap, and cause a sudden change. Based on this simulation, better tracking method can be derived

Fusion of wearable and contactless sensors for intelligent gesture recognition

This paper presents a novel approach of fusing datasets from multiple sensors using a hierarchical support vector machine algorithm. The validation of this method was experimentally carried out using an intelligent learning system that combines two different data sources. The sensors are based on a contactless sensor, which is a radar that detects the movements of the hands and fingers, as well as a wearable sensor, which is a flexible pressure sensor array that measures pressure distribution around the wrist. A hierarchical support vector machine architecture has been developed to effectively fuse different data types in terms of sampling rate, data format and gesture information from the pressure sensors and radar. In this respect, the proposed method was compared with the classification results from each of the two sensors independently. Datasets from 15 different participants were collected and analyzed in this work. The results show that the radar on its own provides a mean classification accuracy of 76.7%, while the pressure sensors provide an accuracy of 69.0%. However, enhancing the pressure sensors’ output results with radar using the proposed hierarchical support vector machine algorithm improves the classification accuracy to 92.5%

Hierarchical sensor fusion for micro-gestures recognition with pressure sensor array and radar

This paper presents a hierarchical sensor fusion approach for human micro-gesture recognition by combining an Ultra Wide Band (UWB) Doppler radar and wearable pressure sensors. First, the wrist-worn pressure sensor array (PSA) and Doppler radar are used to respectively identify static and dynamic gestures through a Quadratic-kernel SVM (Support Vector Machine) classifier. Then, a robust wrapper method is applied on the features from both sensors to search the optimal combination. Subsequently, two hierarchical approaches where one sensor acts as ‛enhancer‚ of the other are explored. In the first case, scores from Doppler radar related to the confidence level of its classifier and the prediction label corresponding to the posterior probabilities are utilized to maximize the static hand gestures classification performance by hierarchical combination with PSA data. In the second case, the PSA acts as an ‛Enhancer‚ for radar to improve the dynamic gesture recognition. In this regard, different weights of the ‛Enhancer‚ sensor in the fusion process have been evaluated and compared in terms of classification accuracy. A realistic cross-validation method is chosen to test one unknown participant with the model trained by data from others, demonstrating that this hierarchical fusion approach for static and dynamic gestures yields approximately 16.7% improvement in classification accuracy in the best cases

Wearable Wristworn Gesture Recognition Using Echo State Network

This paper presents a novel gesture sensing system for prosthetic limb control based on a pressure sensor array embedded in a wristband. The tendon movement which produces pressure change around the wrist can be detected by pressure sensors. A microcontroller is used to gather the data from the sensors, followed by transmitting the data into a computer. A user interface is developed in LabVIEW, which presents the value of each sensor and display the waveform in real-time. Moreover, the data pattern of each gesture varies from different users due to the non-uniform subtle tendon movement. To overcome this challenge, Echo State Network (ESN), a supervised learning network, is applied to the data for calibrating different users. The results of gesture recognition show that the ESN has a good performance in multiple dimensional classifications. For experimental data collected from six participants, the proposed system classifies five gestures with an accuracy of 87.3%

Visual Hand Tracking on Depth Image Using 2-D Matched Filter

Hand detection has been the central attention of human-machine interaction in recent researches. In order to track hand accurately, traditional methods mostly involve using machine learning and other available libraries, which requires a lot of computational resource on data collection and processing. This paper presents a method of hand detection and tracking using depth image which can be conveniently and manageably applied in practice without the huge data analysis. This method is based on the two-dimensional matched filter in image processing to precisely locate the hand position through several underlying codes, cooperated with a Delta robot. Compared with other approaches, this method is comprehensible and time-saving, especially for single specific gesture detection and tracking. Additionally, it is friendly-programmed and can be used on variable platforms such as MATLAB and Python. The experiments show that this method can do fast hand tracking and improve accuracy by selecting the proper hand template and can be directly used in the applications of human-machine interaction. In order to evaluate the performance of gesture tracking, a recorded video on depth image model is used to test theoretical design, and a delta parallel robot is used to follow the moving hand by the proposed algorithm, which demonstrates the feasibility in practice